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FINAL ACTION 

Response to Amendment 

1 . The following Office Action is responsive to the amendment and remarks 
received on August 26, 201 0. Claim 1 8 is added. Claims 1 , 2, 4-7, 9,11,12,14-17 and 
newly added claim 18 remain pending. 



Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 1 02 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious 
at the time the invention was made to a person having ordinary skill in the art to which said 
subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

3. Claims 1, 2, 5-7, 9, 1 1 , 12, 14, 17, and 18 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over the combination of Mitsui (US 7,222,235 B1 ), Barr et al. (US 
2004/0039914 A1), and Rhoads (US 6,411,725 B1). 

Regarding claim 1, Mitsui discloses an information attaching/detecting system 
comprising: 

- object extracting means, for extracting a plurality of regions, from within the 
image (as depicted in figure 15, numeral 106 will perform the object extracting means, 
"[t]he image inputting apparatus 101(m) inputs an image and converts the input image 
to electronic data. The image inputting apparatus 101(m) may be a scanner or a digital 
camera. The image inputting apparatus 101(m) outputs the input image data as an 
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original image A(m). The image processing apparatus 106(m) has a function of inputting 
the original image A(m) from the image inputting apparatus 101(m) and outputting a 
processed image C(m). The image processing apparatus 106(m) generates the 
processed image C(m) by embedding a digital watermark into the original image A(m)" 
at column 22, line 1 1 . The plurality of photographed objects is shown in figure 33, "FIG. 
33 shows how the digital watermark is embedded in the form Y ... the digital watermark 
is shaded for an explanation. For example, different kinds of digital watermarks are 
embedded for each of the region S1 and the region S2 because the character size is 
different between the region S1 and the region S2 ... Furthermore, because the region 
S3 is a photograph, the digital watermark, which is different from in the region S1 and 
the region S2, is embedded in the region S3" at column 38, line 56. See also figure 32 
which indicates a plurality of regions, such as character with REGION 1 and REGION2 
and photograph with REGION 3 and REGION n, is valid for object extraction. Thus, 
Mitsui discloses object extracting means for extracting a plurality of regions from an 
image); 

- information attaching means for attaching different information to each of the 
regions that do not overlap with other regions and acquiring said information-attached 
image (as discussed above and shown in figure 15, wherein numeral 106 will perform 
the information attaching means: "...The image processing apparatus 106(m) generates 
the processed image C(m) by embedding a digital watermark into the original image 
A(m)" at column 22, line 18. Different information attaches to each not overlap regions 
is shown in figure 33: "FIG. 33 shows how the digital watermark is embedded in the 
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form Y ... the digital watermark is shaded for an explanation. For example, different 
kinds of digital watermarks are embedded for each of the region S1 and the region S2 
because the character size is different between the region S1 and the region S2 ... 
Furthermore, because the region S3 is a photograph, the digital watermark, which is 
different from in the region S1 and the region S2, is embedded in the region S3" at 
column 38, line 56. See also figure 32 which indicates a plurality of regions, such as 
character with REGION 1 and REGION2 and photograph with REGION 3 and REGION 
n, is valid for embedding information); 

- detection means for detecting said information from received data for each of 
said plurality regions contained in said information-attached image (as depicted in figure 
15, numeral 108 will detect the embedded information received from transmitted 
information-attached image through network 104. Figure 19 shows a block diagram for 
the information detection. Detailed discussion about the detection processing can be 
found from column 24, line 22 to column 25, line 8). 

Mitsui discloses that not overlapped regions of an image are used for information 
attaching/detecting. Mitsui does not explicitly disclose that the regions are persons 
facial regions. Furthermore, Mitsui discloses the detection means from the transmitted 
information-attached image. Mitsui does not explicitly disclose that the received data 
can be a photographed-image by photographing an image reproducing medium on 
which information-attached image is reproduced. 

Barr, in the field of endeavor of security in watermarking ("for authenticating 
electronic or physical media objects using digital watermarks" in paragraph 9, line 1), 
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teaches the "application of digital watermarking is for the authentication of physical and 
electronic media objects, like images, video, audio, and printed media" in paragraph 6, 
line 1 . "Another way to authenticate the object is to embed information about the object 
or the bearer of the object (e.g., in photo ID or other secure documents)" in paragraph 7, 
line 1 . "One aspect of the invention is a method for creating an identification document 
having a biometric image. This method identifies a feature location in the biometric 
image, generates a digital watermark signal, and embeds the digital watermark signal in 
the biometric image such that the digital watermark location is dependent on the feature 
location" in paragraph 13, line 1 . As shown in figure 3, a person facial photo is used for 
document ID information embedding, "In the case of a photo identification document 
that includes a photo of the bearer's face, added security can be attained by including 
facial biometric information in the digital watermark, which is then embedded in the 
facial photo" in paragraph 70, line 7. Thus, Barr discloses that media objects can be 
used to attach information. And the object can be the facial region of a person for the 
data attachment. Without departing from the spirit and scope of Barr's methodology, 
the plurality of no overlapped regions of an image as taught by Mitsui can be used by 
Barr as no overlapped facial regions for data attachment. 

It would have been obvious at the time the invention was made, that one of 
ordinary skill in the art would have been motivated to include the information 
attaching/detecting system Mitsui made with the application of embedding information 
on person facial region as taught by Barr, in order to enhance the security as stated 
above by Barr. 



Application/Control Number: 10/787,391 Page 6 

Art Unit: 2624 

The combination of Mitsui and Barr does not explicitly disclose that the received 
data can be a photographed-image by photographing an image reproducing medium on 
which information-attached image is reproduced. 

Rhoads, in the field of endeavor of multimedia signal processing ("in particular 
relates to encoding information into and decoding information from video objects" at 
column 1 , line 30), teaches a robust data embedding methodology as shown in figure 2, 
"[i]n FIG. 2, physical objects 200 are pre-watermarked in a manner that survives the 
video capture process 202. For an example of a watermarking process that survives 
digital to analog conversion (e.g., printing a digital image on a physical object), and then 
analog to digital conversion (e.g., capture via a video camera) ... The resulting video is 
then transmitted or broadcast 204" at column 7, line 29. Thus, the watermark can 
survive by printing this watermark embedded digital image and then capture this printed 
image via camera. Without departing from the scope and spirit of Rhoads' 
methodology, a second image captured by the camera can be an image from printing 
which created by Mitsui information attached first image. 

It would have been obvious at the time the invention was made, that one of 
ordinary skill in the art would have been motivated to enhance the information 
attaching/detecting system of the Mitsui and Barr combination, with the application of 
data embedding methodology as taught by Rhoads, so the embedded data can survive 
various scenarios as discussed above. 
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Regarding claim 2, information attaching means is means for acquiring said 
information-attached image by hiddenly embedding said information in said image ("An 
image processing system can be provided such that the image providing apparatus 
recognizes a format of an image file, and provides the image file in which the digital 
watermark is embedded in a part of a region based on the format ..." at Mitsui column 5, 
line 61. See also Rhoads figure 4, numeral 404 "...watermark encoding process 404 
encodes the auxiliary information into the content. ..." at Rhoads column 8, line 13). 

Regarding claim 5, image pick-up means is a camera provided in a portable 
terminal (discussed in claim 1 by Mitsui, digital camera can be the image pick-up 
means). 

Regarding claim 6, said information is location information representing storage 
locations of audio data correlated with said plurality of photographed objects ("Another 
aspect of the invention is a method for using a watermark that has been encoded into a 
video signal ... The watermark may include a direct (e.g., URL or network address) or 
indirect link (e.g., object identifier) to the web site ..." at Rhoads column 2, line 35), 
and which further comprises audio data acquisition means for acquiring said audio data, 
based on said location information (as depicted in Rhoads figure 1 A, numeral 114: "a 
user interface 114 executes and provides visual, audio, or audio-visual information to 
the user ... user interface receives input from the user, selecting a video object. In 
response, it performs an action associated with the selected object using the auxiliary 
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object information decoded from the watermark ..." at Rhoads column 4, line 17. See 
also "The watermark may carry information or programmatic action. It may also link to 
external information or an action, such as retrieval and output of information stored 
elsewhere in a database, website, etc. Watermark linking enables the action associated 
with the watermark to be dynamic ..." at Rhoads column 4, line 29). 

Regarding claim 1 1 , the information device according to claim 1, further 
comprising: 

- a mobile communication terminal having a camera (as discussed in claiml and 
suggested by Rhoads a camera for the mobile communication terminal); 

- a server (as depicted in Rhoads figure 8, numeral 802 is a server); 

- wherein the camera of the mobile terminal photographs the image reproducing 
medium (as discussed in claim 1 and suggested by Rhoads a camera of the mobile 
communication terminal photographs the image reproducing medium); 

- wherein the input means and the detecting means are disposed in the server 
(as depicted in Rhoads figure 8, the server #802 receives input and parses and extracts 
to detect information: "The server, in response to receiving the message (828), parses it 
and extracts an index used to look up a corresponding action in a database (830) that 
associates many such indices to corresponding actions ..." at Rhoads column 14, line 
44). 



Regarding claim 12: - wherein the information is a location of audio data and 
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the server acquires said audio data using said location and transmits said audio data to 
said mobile communication terminal (the combination of Mitsui and Rhoads teaches the 
mobile terminal about the location: "watermark may include a direct (e.g., URL or 
network address) or indirect link (e.g., object identifier) to the web site ..." at Rhoads 
column 2, line 44, to get audio data: "user interface 114 (Rhoads figure 1A) executes 
and provides visual, audio, or audio-visual information to the user ... user interface 
receives input from the user, selecting a video object. In response, it performs an action 
associated with the selected object ..." at column 4, line 17. Furthermore, "The server, 
in response to receiving the message (828) (Rhoads figure8), parses it and extracts an 
index used to look up a corresponding action in a database (830) that associates many 
such indices to corresponding actions ..." at Rhoads column 14, line 44). 

Regarding claim 14: - a region setting section, for setting block regions 
corresponding to each photographed object that do not overlap with other regions that 
include other photographed objects (as discussed in claim 1 , the figure 33 of Mitsui 
contains plurality of not overlapped regions with photographed objects such as S1 , S2, 
S2, and S4); 

- an information attaching section, for attaching different information to each of 
the plurality of block regions which are set by the region setting means (as discussed in 
claim 1, the information attaching means for, information attached to each region is 
different). 
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Regarding claims 7 and 9: - extracting a plurality of facial regions of persons from 
within the image (discussed in claim 1, object extracting means for); 

- attaching different information to each of the plurality of facial regions in said 
image that respectively and acquiring said information-attached image (as discussed in 
claim 1 for information attaching means for. See also Mitsui figure 33, "FIG. 33 shows 
how the digital watermark is embedded in the form Y ... the digital watermark is shaded 
for an explanation. For example, different kinds of digital watermarks are embedded for 
each of the region S1 and the region S2 because the character size is different between 
the region S1 and the region S2 ... Furthermore, because the region S3 is a 
photograph, the digital watermark, which is different from in the region S1 and the 
region S2, is embedded in the region S3" at Mitsui column 38, line 56); 

- receiving photographed-image data obtained by photographing an image 
reproducing medium, on which the information-attached image is reproduced, with 
image pick-up means (as discussed in claim 1, input means for); 

- detecting said information from said photographed-image data for each of said 
plurality of facial regions contained in said information-attached image (as discussed in 
claiml, detection means for); 

- outputting the information corresponding to each of the plurality of facial 
regions, based on the detected information ("Wherever the video is distributed, a 
decoding process may be inserted to decode information about the video object from a 
watermark embedded in the video signal. This information may then be used to trigger 
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an action, such as fetching graphics and displaying it to the user ..." at Rhoads column 
4, line 61); 

- wherein the information attached related to a person corresponding to the one 
of the facial regions to which the information is attached (discussed in claim 1 by Barr 
for person facial region with embedded information). 

Regarding claim 17, wherein the outputting comprises providing a visual or audio 
display via at least one of a audio device and a display monitor (as discussed in claim 7 
for outputting detected information "may then be used to trigger an action, such as 
fetching graphics and displaying it to the user ..." at Rhoads column 4, line 64). 

Regarding claim 18, different facial regions corresponding to different individuals 
appear in the image (as discussed in claim 1 by the combination of Mitsui and Barr 
wherein Mitsui teaches the concept of having plurality of not overlapped different 
regions in an image and Barr teaches the concept of object can be a person's facial 
region for data attachment). 

4. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Mitsui, Barr, and Rhoads as applied to claim 1 discussed above, and 
further in view of Motta et al. (US 6,726,103 B1). 
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Regarding claim 4, the Mitsui, Barr, and Rhoads combination discloses an 
information attaching/detecting system with image pick-up means to receive 
photographed-image data. 

The Mitsui, Barr, and Rhoads combination does not explicitly disclose a way to 
correct the geometrical distorted photographed-image data. 

Motta, in the same field of endeavor of digital imaging ("relates to the field of 
image sensors and imaging systems" at column 1 , line 8), teaches a geometric 
correction processing system for the photographed-image data as depicted in figure 3. 
"Data processing may include but is not limited to data interpolation, noise reduction, 
color adjustment, and/or geometric corrections due to optical aberrations ..." at column 

4, line 4. 

It would have been obvious at the time the invention was made, that one of 
ordinary skill in the art would have been motivated to provide the information 
attaching/detecting system made by Mitsui, Barr, and Rhoads combination with the 
geometrical distortion correction processor as taught by Motta, in order to provide "[a] 
system that can self test, detect errors, and correct such errors ... and would be of great 
value to many businesses and industries" at Motta column 2, line 29. 

5. Claims 15 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over the combination of Mitsui, Barr, and Rhoads as applied to claim 1 discussed 
above, and further in view of Hsieh et al. (US 7,003,135 B2). 
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Regarding claims 15 and 16, the Mitsui, Barr, and Rhoads combination discloses 
an information attaching/detecting system to extract a plurality of facial regions of 
persons from within the image. 

The Mitsui, Barr, and Rhoads combination does not explicitly disclose skin 
color/face contours can be used for facial region extraction. 

Hsieh, in the field of endeavor of faces tracking ("which is able to effectively 
detect and track a plurality of faces in real-time" at column 2, line 7), based on the fact 
that "[i]n order to efficiently separate the face regions from the complex backgrounds, a 
preferable approach is to use the hybrid information of color and motion. Then, the 
segmented regions are further verified to determine whether the correct faces are 
extracted from the remaining distracters" at column 1 , line 42, teaches a methodology to 
track no-overlap human faces as shown in figure 1 , numeral 10 and overlapped faces 
with: "a face-like region generator having a skin region extractor, a motion analyzer and 
a silhouette analyzer. The skin region extractor generates a plurality of skin regions by 
detecting skin color pixels of an input image. The motion analyzer determines possible 
face-like regions from the skin regions based on moving information of the input image. 
The silhouette analyzer analyzes whether there exists a raised shape in the image so 
as to separate connected regions ..." at column 2, line 39. Thus, Hsieh discloses a 
methodology to detect regions which are person facial regions for real-time tracking by: 

a) Skin region extractor for facial regions to generate skin-color regions by detecting 
skin color pixels of an input image (claim 15). 

b) Silhouette analyzer for facial regions to generate face contours (claim 1 6). 
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It would have been obvious at the time the invention was made, that one of 
ordinary skill in the art would have been motivated to include the information 
attaching/detecting system made by Mitsui, Barr, and Rhoads combination with the 
application of face-like region generator as taught by Hsieh, in order to effectively 
tracking human faces in real-time as stated above by Hsieh. 

Response to Arguments 

A) Summarv of Applicant's Remark : 

The previous claim USC § 101 rejections should be withdrawn in view of the 
amendment. 
Examiner's Response : 

Examiner agrees, and the previous USC § 101 rejections are withdrawn. 
Please refer to the rejections above for further discussion. 

B) Summary of Applicant's Remark : 

"Applicants respectfully submit that even if combined as suggested, the 
suggested combination fails to disclose "detecting said information from said 
photographed-image data for each of said plurality of facial regions," and "attaching 
means for attaching different information to each of the facial regions," as recited in 
claim 1" at Remarks page 7, line 22. 

"However, the Barr reference does not disclose detecting facial regions as 
required in Claim 1 . Further, the Barr reference is silent regarding the technical concept 
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of extracting a plurality of facial regions , and attaching different information to each 
extracted facial region . That is, the Barr reference discloses only that data related to a 
single person is embedded in an image of the person as an electronic watermark, and is 
silent regarding the technical concept of attaching different pieces of information related 
to a plurality of people respectively to the facial regions of the people which are included 
in a single image" at Remarks page 8, line 1 1 . 
Examiner's Response : 

Mitsui teaches the concepts of having plurality of no overlapped regions for 
attaching different information. Barr teaches the concepts of media object can be the 
facial region of a person for data attachment. It is the combination of Mitsui and Barr 
teaches the claimed subject matter. Please refer to the rejections above for further 
discussion. 

Conclusion 

6. Applicant's amendment is rejected in this Office action. Accordingly, THIS 
ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the 
extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
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the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Eueng-nan Yeh whose telephone number is 571-270- 
1586. The examiner can normally be reached on Monday-Friday 8AM-4:30PM EDT. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on 571-272-7415. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 

Patent Application Information Retrieval (PAIR) system. Status information for 

published applications may be obtained from either Private PAIR or Public PAIR. 

Status information for unpublished applications is available through Private PAIR only. 

For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 

you have questions on access to the Private PAIR system, contact the Electronic 

Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 

USPTO Customer Service Representative or access to the automated information 

system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Eueng-nan Yeh 
Patent Examiner 
Art Unit: 2624 
/E.Y./ 
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